10 research outputs found
Robust Multi-bit Natural Language Watermarking through Invariant Features
Recent years have witnessed a proliferation of valuable original natural
language contents found in subscription-based media outlets, web novel
platforms, and outputs of large language models. However, these contents are
susceptible to illegal piracy and potential misuse without proper security
measures. This calls for a secure watermarking system to guarantee copyright
protection through leakage tracing or ownership identification. To effectively
combat piracy and protect copyrights, a multi-bit watermarking framework should
be able to embed adequate bits of information and extract the watermarks in a
robust manner despite possible corruption. In this work, we explore ways to
advance both payload and robustness by following a well-known proposition from
image watermarking and identify features in natural language that are invariant
to minor corruption. Through a systematic analysis of the possible sources of
errors, we further propose a corruption-resistant infill model. Our full method
improves upon the previous work on robustness by +16.8% point on average on
four datasets, three corruption types, and two corruption ratios. Code
available at https://github.com/bangawayoo/nlp-watermarking.Comment: ACL 2023 lon
Self-Distilled Self-Supervised Representation Learning
State-of-the-art frameworks in self-supervised learning have recently shown
that fully utilizing transformer-based models can lead to performance boost
compared to conventional CNN models. Striving to maximize the mutual
information of two views of an image, existing works apply a contrastive loss
to the final representations. Motivated by self-distillation in the supervised
regime, we further exploit this by allowing the intermediate representations to
learn from the final layer via the contrastive loss. Through self-distillation,
the intermediate layers are better suited for instance discrimination, making
the performance of an early-exited sub-network not much degraded from that of
the full network. This renders the pretext task easier also for the final
layer, lead to better representations. Our method, Self-Distilled
Self-Supervised Learning (SDSSL), outperforms competitive baselines (SimCLR,
BYOL and MoCo v3) using ViT on various tasks and datasets. In the linear
evaluation and k-NN protocol, SDSSL not only leads to superior performance in
the final layers, but also in most of the lower layers. Furthermore, positive
and negative alignments are used to explain how representations are formed more
effectively. Code will be available.Comment: 15 page
Model Compression via Position-Based Scaled Gradient
We propose the position-based scaled gradient (PSG) that scales the gradient depending on the position of a weight vector to make it more compression-friendly. First, we theoretically show that applying PSG to the standard gradient descent (GD), which is called PSGD, is equivalent to the GD in the warped weight space, a space made by warping the original weight space via an appropriately designed invertible function. Second, we empirically show that PSG acting as a regularizer to the weight vectors is favorable for model compression domains such as quantization, pruning, and knowledge distillation. PSG reduces the gap between the weight distributions of a full-precision model and its compressed counterpart. This enables the versatile deployment of a model either as an uncompressed mode or as a compressed mode depending on the availability of resources. The experimental results on CIFAR-10/100 and ImageNet datasets show the effectiveness of the proposed PSG in model compression including an iterative pruning method and the knowledge distillation
Inactivating transcription factor OsWRKY5 enhances drought tolerance through abscisic acid signaling pathways
During crop cultivation, water-deficit conditions retard growth, thus reducing crop productivity. Therefore, uncovering the mechanisms behind drought tolerance is a critical task for crop improvement. Here, we show that the rice (Oryza sativa) WRKY transcription factor OsWRKY5 negatively regulates drought tolerance. We determined that OsWRKY5 was mainly expressed in developing leaves at the seedling and heading stages, and that its expression was reduced by drought stress and by treatment with NaCl, mannitol, and abscisic acid (ABA). Notably, the genome-edited loss-of-function alleles oswrky5-2 and oswrky5-3 conferred enhanced drought tolerance, measured as plant growth under water-deficit conditions. Conversely, the overexpression of OsWRKY5 in the activation-tagged line oswrky5-D resulted in higher susceptibility under the same conditions. The loss of OsWRKY5 activity increased sensitivity to ABA, thus promoting ABA-dependent stomatal closure. Transcriptome deep sequencing and reverse transcription quantitative polymerase chain reaction analyses demonstrated that the expression of abiotic stress-related genes including rice MYB2 (OsMYB2) was upregulated in oswrky5 knockout mutants and downregulated in oswrky5-D mutants. Moreover, dual-luciferase, yeast one-hybrid, and chromatin immunoprecipitation assays showed that OsWRKY5 directly binds to the W-box sequences in the promoter region of OsMYB2 and represses OsMYB2 expression, thus downregulating genes downstream of OsMYB2 in the ABA signaling pathways. Our results demonstrate that OsWRKY5 functions as a negative regulator of ABA-induced drought stress tolerance, strongly suggesting that inactivation of OsWRKY5 or manipulation of key OsWRKY5 targets could be useful to improve drought tolerance in rice cultivars.Y
The Rice CHD3/Mi-2 Chromatin Remodeling Factor Rolled Fine Striped Promotes Flowering Independent of Photoperiod
Genetic studies have revealed that chromatin modifications affect flowering time, but the underlying mechanisms by which chromatin remodeling factors alter flowering remain largely unknown in rice (Oryza sativa). Here, we show that Rolled Fine Striped (RFS), a chromodomain helicase DNA-binding 3 (CHD3)/Mi-2 subfamily ATP-dependent chromatin remodeling factor, promotes flowering in rice. Diurnal expression of RFS peaked at night under short-day (SD) conditions and at dawn under long-day (LD) conditions. The rfs-1 and rfs-2 mutants (derived from different genetic backgrounds) displayed a late-flowering phenotype under SD and LD conditions. Reverse transcription-quantitative PCR analysis revealed that among the flowering time-related genes, the expression of the major floral repressor Grain number and heading date 7 (Ghd7) was mainly upregulated in rfs mutants, resulting in downregulation of its downstream floral inducers, including Early heading date 1 (Ehd1), Heading date 3a (Hd3a), and Rice FLOWERING LOCUS T 1 (RFT1). The rfs mutation had pleiotropic negative effects on rice grain yield and yield components, such as plant height and fertility. Taking these observations together, we propose that RFS participates in multiple aspects of rice development, including the promotion of flowering independent of photoperiod